A Haptic-audio Human-Computer Interface

For the visually impaired it is important to acquire knowledge about environments in advance. Haptic-audio interfaces have been proven as appropriate to represent certain types of spatial knowledge in a non-visual manner. This dissertation elaborates on a haptic-audio interface that supports non-visual spatial knowledge acquisition of indoor environments. Three representational layers relevant for acquisition of indoor spatial knowledge using the haptic-audio interface are defined in the following: (a) knowledge concerning room access and the layout of individual rooms and whole flats, (b) knowledge about the location of functional facilities (such as, windows, radiators, and power-outlets), and (c) knowledge regarding the position of furniture. In correspondence, three research questions were investigated, namely (i) how to represent boundaries among areas and access to areas, (ii) how to represent linear spatial overlapping aroused by the presence of functional facilities, and (iii) how to represent 2-D spatial overlapping aroused by furniture pieces in rooms. Based on these considerations, a haptic-audio interface is implemented as non-visual floor plans that can be perceived with the PHANToM virtual force feedback device. In order to discuss the research questions mentioned above, a user-centered research methodology is employed in this dissertation. User behavior was observed and evaluated with help of usability studies. Users requirements and expectations were investigated through focus group studies. Based on empirical analyses of the three representation layers of the haptic-audio interface, this dissertation represents room access and layout of an indoor environment through virtual haptic models. Linear spatial overlaps are represented with the support of sonification. Further more, a two-phase haptic-audio exploration strategy is proposed to acquire spatial knowledge involving 2-D spatial overlapping: In the first phase, the floor plan on display is unfurnished and the functional facilities are sonified. Then in the second phase, virtualization of furniture pieces is further displayed. The empirical results show that spatial knowledge of room-access and layouts of indoor environments can be acquired by visually impaired people through exploration in haptic-audio virtual environment. Plausibility of acquiring spatial knowledge involving linear overlapping and 2-D overlapping is also presented.